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1  1  1 


ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&O  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  purpose  of  this  research  was  to  identify  effective  features  of  perfor¬ 
mance  appraisal  systems  that  might  be  considered  for  inclusion  in  the  Depart¬ 
ment  of  the  Army's  performance  appraisal  system  for  civilian  personnel.  Since 
this  system  is  required  by  law  to  meet  the  same  design  and  functional  require¬ 
ments  as  other  federal  performance  appraisal  systems  covered  by  the  Civil  Ser¬ 
vice  Reform  Act  of  1978,  the  approach  taken  was  to  both  review  the  literature 
on  performance  appraisal  and  to  interview  subject-matter  experts  of  other  fed¬ 
eral  agencies  in  order  to  learn  about  particularly  effective  features  of  their 
systems.  This  report  is  likely  to  be  of  interest  to  both  civilian  and  military 
managers,  supervisors,  and  personnelists ,  within  the  Department  of  the  Army  and 
in  other  federal  agencies,  as  it  is  concerned  with  performance  appraisal,  per¬ 
formance  management,  and  accomplishment  of  organizational  goals. 


RECOMMENDATIONS  FOR  IMPROVED  PERFORMANCE  APPRAISAL  IN  THE  FEDERAL  SECTOR 


EXECUTIVE  SUMMARY 


Requirement : 

To  identify  those  features  of  performance  appraisal  systems  that  are  par¬ 
ticularly  effective  and  that  also  meet  the  legal  requirements  for  federal  agen¬ 
cies  covered  by  the  Civil  Service  Reform  Act  of  1978. 


Procedure : 

Two  approaches  were  used.  The  first  was  a  review  of  the  performance  ap¬ 
praisal  literature,  with  special  emphasis  on  performance  appraisal  systems,  or 
features  of  systems,  which  are  consistent  with  the  Civil  Service  Reform  Act  re¬ 
quirements.  The  literature  included  standard,  current  reference  works  in  the 
field  of  performance  appraisal,  relevant  original  sources  cited  in  these  refer¬ 
ences,  and  all  relevant  articles  from  10  personnel/management  Journals  from 
1982  through  the  present. 

Since  very  little  literature  is  available  on  current  federal  performance 
appraisal  systems,  a  second  approach  was  used  to  obtain  specific  information 
about  effective  features  of  these  systems.  More  than  100  interviews  were  con¬ 
ducted  with  federal-sector  subject-matter  experts  who  had  roles  in  developing, 
administering,  monitoring,  and/or  evaluating  their  agencies'  systems. 


Findings : 

Nine  prescriptive  themes  for  more  effective  federal  performance  appraisal 
systems  are  discussed.  The  themes,  which  emphasized  the  importance  of  using  a 
systems  approach  in  making  performance  appraisal  work  as  intended,  were  as  fol¬ 
lows:  (l)  legal  requirements  must  be  met,  (2)  performance  appraisal  needs  to 

be  viewed  as  important  and  worthwhile  by  users,  (3)  top  management  backing  is 
essential,  (U)  the  performance  appraisal  process  should  be  used  as  a  management 
tool,  (5)  the  performance  appraisal  process  should  be  linked  to  organizational 
goals,  (6)  the  performance  appraisal  system  should  be  adapted  to  organizational 
needs,  (7)  system  perpetuation  should  be  planned,  (8)  training  is  essential, 
and  (9)  ease  of  use  Is  important. 


Utilization  of  Findings: 

The  findings  of  this  study  should  be  useful  to  management,  supervisory,  an 
personnel  audiences  In  federal  agencies  covered  by  the  Civil  Service  Reform  Act 
of  1978.  They  will  enable  these  audiences  to  understand  their  role  in  the  fed¬ 
eral  performance  appraisal  system,  how  the  system  can  facilitate  the  accomplish 
ment  of  organizational  goa’.  s,  and  what  is  needed  to  make  It  work. 
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RECOMMENDATIONS  FOR  IMPROVED  PERFORMANCE  APPRAISAL 
IN  THE  FEDERAL  SECTOR 


INTRODUCTION 

The  Civil  Service  Reform  Act  (CSRA)  of  1978  directed  specified  Federal 
agencies,  including  the  Department  of  the  Army,  to  establish  performance 
appraisal  (PA)  systems  that  would  be  used  as  a  basis  for  training,  rewarding, 
promoting,  and  removing  employees  as  well  as  other  personnel  actions.  In 
compliance  with  specific  provisions  of  CSRA,  the  Department  of  the  Army 
established  the  required  PA  systems  for  its  various  categories  of  civilian 
personnel  (Army  Regulation  No.  690-900,  690-500,  and  690-400). 

Informal  feedback  from  within  the  Department  of  the  Army  has  revealed  a 
lack  of  enthusiasm  for  the  current  PA  systems.  Army  civilian  PA  is  seen  by 
many  as  overly  complex,  too  time-consuming,  and  of  uncertain  value. 
Consequently,  the  Director  of  Civilian  Personnel  for  the  Army  (Office  of  the 
Deputy  Chief  of  Staff  for  Personnel),  requested  that  the  U.S.  Army  Research 
Institute  (ARI)  conduct  research  to  identify  the  parameters  of  an  improved  PA 
system  for  the  Army. 

The  research  conducted  by  ARI  in  response  to  this  request  was  designed  to 
identify  those  features  of  PA  systems  which  have  been  particularly  effective 
and  which  also  meet  the  legal  requirements  tor  Federal  agencies  covered  by 
CSRA.  Three  approaches  were  used  to  accomplish  this:  a  review  of  the  PA 
literature,  a  survey  to  identify  successful  features  of  current  Federal 
agency  PA  6y6tems  (see  Steinberg  &  Burke,  in  press),  and  Interviews  with 
subject-matter  experts. 

This  report  is  divided  into  two  parts.  The  first  i6  a  method  section 
that  describes  the  literature  review  and  Interviews  with  subject-matter 
experts  and  the  second  is  a  synthesis  of  the  findings  organized  around  nine 
themes . 


METHOD 

This  section  describes  the  two  sources  of  information  upon  which  the 
themes  and  conclusions  presented  in  later  sections  are  based.  The  two 
sources  are  selections  from  the  PA  literature  and  interviews  with 
6ubject-matter  experts. 

Literature  Review 


The  literature  in  the  field  of  PA  currently  is  vast,  and  is  growing  at  a 
rapid  rate.  Literally  thousands  of  articles  have  been  published  on  the  topic 
within  the  last  30  years  (Henderson,  1984).  Consequently,  an  attempt  was 
made  to  limit  the  review  to  literature  emphasizing  systems,  or  features  of 
systems,  consistent  with  CSRA  requirements. 


I 


The  literature  reviewed  Includes  a  mix  of  empirical  studies  and  expert 
opinion.  Although  the  latter  may  be  based  on  untested  hypotheses,  it  is 
included  because  it  represents  the  recommendations  of  experts  in  the  field. 

The  review  began  with  standard,  current  reference  works  in  the  field  of 
PA  (e.g.,  Baird,  Beatty,  &  Schneier,  1982;  Bernardin  &  Beatty,  1984; 
Conference  Board,  1979;  DeVries,  Morrison,  Shullman,  &  Gerlach,  1980;  Landy  8 
Farr,  1980;  Latham  &  Vexley,  1981;  McCall  8  DeVries,  1977).  Then,  original 
sources  cited  in  these  references  were  reviewed.  Their  selection  was  based 
on  the  relevance  of  their  cited  titles  and  content  or  their  repeated  mention 
in  the  literature.  These  sources,  in  turn,  often  suggested  other  pertinent 
references.  In  addition,  recent  (1982  through  the  present)  volumes  of 
frequently  cited  journals  were  searched  for  relevant  articles.  These 
journals  included  the  following: 

o  Academy  of  Management  Journal 

o  Academy  of  Management  Proceedings 

o  Academy  of  Management  Review 

o  Journal  of  Applied  Psychology 

o  Management 

o  Management  Review 

o  Personnel  Journal 


o  Personnel  Psychology 
o  Public  Personnel  Management 
o  Training  and  Development  Journal 
Interviews  with  Subject-Matter  Experts 

Very  little  literature  is  available  on  current  Federal  PA  systems. 
Therefore,  a  survey  of  Federal  agency  subject-matter  experts  was  conducted  in 
order  to  learn  about  these  systems.  The  survey  procedures  are  described 
elsewhere  (Steinberg  &  Burke,  in  press)  but  the  results  are  considered  here 
in  conjunction  with  the  rest  of  the  literature  reviewed.  Its  relevance  in 
this  section  is  that  in  the  process  of  developing  and  administering  the 
survey,  and  also  subsequent  to  its  administration,  both  formal  and  informal 
interviews  were  conducted  with  more  than  100  subject-matter  experts.  These 
interviews  are  considered  another  source  of  information  for  the  purposes  of 
the  present  report. 

The  surveys  were  mailed  to  the  members  of  the  Committee  on  Performance 
Management  of  the  Office  of  Personnel  Management  (0PM)  Interagency  Advisory 
Group  (IAG).  The  members  of  that  committee,  each  representing  a  different 
Federal  agency,  play  a  major  role  with  respect  to  their  agencies'  PA  systems. 


They  have  responsibilities  for  developing,  administrating,  monitoring,  and/or 
evaluating  their  agencies'  systems.  Their  job  titles  include  Program 
Manager,  Personnel  Management  Specialist,  Agency  Coordinator,  Director  of  the 
Office  of  Performance  Evaluation,  Chief  of  Personnel  Programs  and  Policy,  and 
Performance  Systems  Officer. 

The  interviews  were  conducted  in  the  following  contexts: 

o  Federal  sector  personnel  managers  intimately  knowledgeable  about  PA 
were  consulted  for  background  information  utilized  in  the  design  of 
the  survey. 

o  Additional  Federal  sector  subject-matter  experts  participated  in  the 
pretest  of  the  survey.  During  these  sessions,  participants  related 
their  ideas  and  experiences. 

o  Telephone  contact  was  made  with  each  of  the  88  IAG  committee  members 
to  enlist  participation  in  the  survey.  Many  used  this  opportunity 
to  air  their  concerns  and  recommendations  regarding  PA. 

o  Followup  Interviews  were  conducted  with  a  number  of  survey 
respondents  to  obtain  more  detailed  Information  about  their 
recommendations  and  also  with  other  subject-matter  experts 
recommended  by  the  survey  respondents. 

Note  that  the  interviews  with  subject-matter  experts  were  not  used  to 
evaluate  research  .ypo  theses.  Rather,  they  were  used  to  elicit  thoughts  and 
ideas  based  on  experience  in  the  PA  arena. 

LESSONS  LEARNED 

Nine  prescriptive  themes  emerged  from  the  integration  of  the  literature 
review  and  the  interviews.  These  themes  summarize  the  parameters  of  an 
effective  PA  system  for  Federal  agencies  under  CSRA.  These  themes  are  as 
follows : 

A  PA  system  appears  to  work  when: 
o  legal  requirements  are  met. 

o  PA  is  viewed  as  important  and  worthwhile  by  the  users, 
o  top  management  barks  it. 
o  appraisal  is  used  as  a  management  tool, 
o  appraisal  is  linked  to  organizational  goals, 
o  it  is  adapted  to  organizational  needs. 


The  first  lesson  learned  from  both  the  literature  review  and  the 
interviews  is  that  a  PA  system  must  be  designed  with  legal  requirements  in 
mind.  Appraisal  systems  are  being  challenged  in  the  courts  with  increasing 
frequency.  This  has  led  to  an  increased  awareness  of,  and  attempts  to  comply 
with,  the  legal  constraints  within  which  an  appraisal  system  should  be 
designed  (Edwards,  1983;  Schneier  &  Beatty,  1979).  This  awareness  of  legal 
constraints  is  heightened  by  the  fact  that  the  cost  to  agencies  found  in 
violation  of  the  law  can  easily  run  into  millions  of  dollars  (Latham  & 

Uexley,  1981).  Three  major  pieces  of  legislation  affecting  the  design  of  PA 
systems  in  the  federal  sector  are  discussed  below. 

Title  VII  of  the  Civil  Rights  Act  of  1964.  Title  VII  of  the  Civil  Rights 
Act  of  1964  (P.L.  88-352)  prohibits  discrimination  in  employment  decisions. 
The  result  of  court  actions  initiated  under  the  provisions  of  this  act  have 
made  it  clear  that  PA  is  among  the  employment  practices  covered  by  it 
(Lubben,  Thompson,  &  Klasson,  1980).  Further,  the  Uniform  Guidelines  on 
Employee  Selection  Procedures  (1978)  provide  that  if  an  employment  practice 
(e.g.,  PA)  is  shown  to  have  adverse  impact,  the  employer  must  demonstrate 
that  the  employment  practice  has  been  validated  in  accordance  with  the 
Uniform  Guidelines.  This  validation  must  begin  with  a  formal  job  analysis  of 
the  position  in  question  (Lubben  et  al . ,  1980).  “There  should  be  a  review  of 
job  information  to  determine  measures  of  work  behaviors  or  performance  that 
are  relevant  to  the  extent  that  they  represent  critical  or  important  Job 
duties,  work  behaviors  or  work  outcomes  as  developed  from  the  review  of  job 
information.  In  view  of  the  possibility  of  bias  in  subjective  evaluations, 
supervisory  rating  techniques  should  be  carefully  developed;  and  the  ratings 
should  be  examined  for  evidence  of  racial,  ethnic  or  sex  bias"  (Uniform 
Guidelines  on  Employee  Selection  Procedures,  1978,  p.  38300). 

Prescriptions  for  PA  systems  typically  include  the  recommendation 
for  a  job  analysis  not  only  as  sound  personnel  practice,  but  also  as  a 
precautionary  measure  in  case  the  system  is  challenged  (Bernardin  &  Beatty, 
1984;  Heneman,  Schwab,  Fossum,  &  Dyer,  1983;  Lubben  et  al . ,  1980).  (A  Pa 
system  lacking  certain  critical  features,  which  include  a  job  analysis,  has 
been  equaled  to  “a  cocked  gun  pointed  at  the  employer"  Landy,  1985,  p.  171.) 
Indeed,  a  court  challenge  may  occur  even  in  the  absence  of  adverse  impact. 

As  Klelman  and  Durham  in  their  review  of  23  Title  VII  court  cases  have 
concluded:  "the  court  has  shown  Interest  in  assessing  performance  appraisal 
systems  regardless  of  their  adverse  impact"  (1981,  p.  103). 
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Civil  Service  Reform  Act  of  1978.  The  Civil  Service  Reform  Act  of  1978 
(P.L~  95-454)  and  the  OPM  regulations  which  implement  it  (Office  of  Personnel 
Management,  1979)  call  for  PA  systems  which  include  the  following  features: 

o  At  the  beginning  of  each  appraisal  period,  a  performance  plan  should 
be  written.  This  plan  will  contain: 

the  critical  elements  of  the  employee's  job  (i.e.,  duties  and 
responsibilities  of  the  Job  for  which  performance  below  the 
minimum  standard  requires  remedial  action  and  denial  of  a 
withln-grade  increase),  and 

-  the  standards  by  which  performance  on  these  critical  elements 
will  be  evaluated.  The  performance  standards  must  be  objective 
criteria  related  to  the  duties  and  responsibill ties  of  the  job. 
They  may  Include,  but  are  not  limited  to,  factors  such  as 
quantity,  quality,  and  timeliness. 

o  Employees  are  encouraged  to  participate  in  the  establishment  of  the 
performance  standards. 

o  No  preestablished  rating  distribution  is  permitted,  such  as  a  forced 
distribution  on  a  bell  curve.  Appraisal  must  be  based  on  actual 
performance  on  the  Job  and  preestablished  objective  standards  for 
performance . 

Civil  Service  Retirement  Spouse  Equity  Act  of  1984.  Title  II  of  the 
Civil  Service  Retirement  Spouse  Equity  Act  of  1984  (P.L.  98-615)  amends 
portions  of  Title  V  of  the  United  States  Code,  dealing  with  PA  systems.  (The 
Title  II  portion  of  the  act  i6  based  largely  on  S.958,  proposed  Civil  Service 
Amendments  of  1984,  and  does  not  deal  with  retirement  or  spouse  equity.)  Thi6 
legislation  and  the  regulations  that  implement  it  mandated  the  Performance 
Management  and  Recognition  Sys tern  (PURS)  to  replace  the  Merit  Pay  system  for 
Federal  supervisors  and  managers,  and  modified  and/or  added  specific  features 
to  the  provisions  of  CSRA.  These  are  as  follows  (Office  of  Personnel 
Management,  1985): 

o  Training  and  Information  is  now  required  for  PMRS  employees  and 
their  supervisors  on  the  purpose  of  the  Performance  and  Management 
Recognition  system  and  how  it  works. 

o  Uniform  five-tiered  summary  rating  levels  for  PMRS  employees  are 
specified.  They  are  "Fully  Successful,"  with  two  levels  above  it, 
6uch  as  "Exceeds  Fully  Successful”  and  "Outstanding,"  and  two  levels 
below  it,  such  as  "Minimally  Successful"  and  "Unacceptable" . 

o  Performance  of  PMRS  employees  on  assigned  details  of  120  or  more 
days  within  or  outside  the  agency  should  be  taken  into  account  in 
determining  ratings.  If  details  are  within  the  agency,  elements  and 
standards  must  be  written  for  It. 
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o  PMRS  appraisal  periods  must  end  not  earlier  than  June  30,  and  not 
later  than  September  30  of  each  year. 

o  Joint  participation  between  PMRS  employees  and  their  supervisors  in 
developing  performance  plans  i6  required.  (For  non-PMRS  employees, 
it  remains  "encouraged.") 

o  PMRS  performance  plans  must  be  reviewed  at  a  higher  level  in  the 
organization  and  higher-level  review  of  appraisals  prior  to 
providing  ratings  to  PMRS  employees  i6  required. 

o  A  progress  review  must  be  held  at  least  once  during  the  appraisal 
period  for  PMRS  employees. 

o  One  or  more  6-member  performance  standards  review  boards  should  be 
established  for  each  agency  to  review  representative  performance 
plans  for  quality  of  the  plans  and  difficulty  of  the  performance 
standards  and  to  report  and  make  recommendations  to  the  head  of  the 
agency  or  designee.  One  half  of  the  board  roust  be  composed  of  PMRS 
employees. 

Although  the  above  items  refer  specifically  to  PMRS  employees,  the 
regulations  emphasize  performance  management  and  the  linkage  of  PA  to 
the  accomplishment  of  organizational  goals  for  non-PMRS  employees  as  well. 

The  term  "performance  management"  is  introduced  and  replaces  the  term 
"performance  appraisal"  in  portions  of  the  regulations.  Although 
"performance  management"  is  not  defined  separately,  the  term  "performance 
management  plan"  is.  That  is  the  "agency's  methods  which  integrate 
performance,  pay,  and  incentive  systems  with  its  basic  management  functions 
for  the  purpose  of  improving  individual  and  organizational  effectiveness  in 
the  accomplishment  of  agency  mission  and  goals"  (Office  of  Personnel 
Management,  1983).  Thus,  the  performance  management  system  is  an  Integrated 
management  system;  and  the  performance  management  plan  includes  the  PA  plan. 

The  emphasis  on  the  linkage  of  PA  to  the  accomplishment  of  organizational 
goals  also  can  be  clearly  seen  in  the  new  Subpart  D  which  has  been  added  to 
the  0PM  regulations  which  Implement  CSRA  (Office  of  Personnel  Management, 
1985).  The  stated  purpose  of  this  subpart  entitled  "Performance  Appraisal 
for  the  Performance  Management  and  Recognition  System"  is  to  "ensure  that 
performance  appraisal  systems  for  PMRS  employees  are  used  as  a  tool  for 
executing  basic  management  and  supervisory  responsibilities  by: 

(a)  communicating  and  clarifying  agency  goals  and  objectives;  (b)  identifying 
individual  accountability  for  the  accomplishment  of  organizational  goals  and 
objectives;  (c)  evaluating  and  improving  individual  and  organizational 
accomplishments"  (p.  11791).  The  subpart  requires  that  accomplishment  of 
organizational  objectives  be  included  in  PMRS  performance  plans  by 
"incorporating  objectives,  goals,  program  plans,  work  plans,  or  other  similar 
means  that  account  for  program  results"  ( p.  11792). 

Differing  Legal  Requirements  for  Civilian  and  Military  Personnel. 

Although  it  has  been  suggested  by  some  that  the  Army  Officer  Evaluation 
Report  (OER)  might  stand  as  a  model  for  a  civilian  PA  system,  a  civilian 
system  cannot  be  adapted  readily  from  the  military  due  to  the  differing 


6 


requirements  of  legislation  that  cover  each  of  these  groups.  Title  VII  of 
the  Civil  Rights  Act  of  1964  has  been  extended  by  the  Equal  Employment 
Opportunity  Act  (1972)  to  govern  both  public  and  private  sectors  (Bernardin 
&  Beatty,  1984).  However,  by  ruling  of  the  Department  of  Defense  General 
Council,  Title  VII  does  not  cover  military  personnel  (Eitelberg,  Laurence, 
Waters,  &  Perelman,  1984).  Also,  members  of  the  uniformed  services  are  not 
covered  by  CSRA,  since  they  are  not  considered  "employed  in  or  under  an 
agency"  (Performance  Rating,  1977,  p.  549).  Instead,  active  duty  military 
personnel  are  governed  by  Title  X  of  the  United  States  Code  (Eitelberg  et 
al.,  1984). 

The  PA  systems  for  civilians  and  military  personnel  within  the  Department 
of  the  Army  reflect  these  different  legal  philosophies  (see  Table  1).  The 
civilian  system  emphasizes  the  evaluation  of  the  employees'  behavior  against 
the  specific  requirements  of  the  job,  whereas  the  military  system  evaluation 
incorporates  attributes  and  competencies  of  the  individual  and  comparison  of 
the  individual  with  others  of  the  same  rank  (Department  of  the  Army,  1985; 
Pelissero,  1984).  Thus,  for  example,  a  GS-11  engineer  is  rated  against  the 
Job  requirements  and  performance  standards  for  a  particular  Job  in  a  given 
department  at  a  given  installation,  whereas  an  Army  Captain  engineer  doing 
the  exact  same  job  is  evaluated  not  only  on  performance  of  duty  requirements 
(no  prior  standards  provided),  but  also  on  (a)  potential  compared  to  all 
other  Army  Captains,  and  (b)  competencies  and  attributes  such  as 
adaptability,  integrity,  moral  courage  (see  Table  2).  In  the  the  military 
system,  no  guidance  is  provided  in  the  regulations  on  what  is  meant  by  these 
latter  dimensions  (Department  of  the  Army,  1985).  Supervisors  provide  their 
own  Interpretations  of  "moral  standards."  In  the  civilian  system  under  CSRA, 
in  order  to  evaluate  competencies  and  attributes,  these  must  be  stated  in 
behavioral  terms  and  shown  to  be  specifically  related  to  the  requirements  of 
the  job.  Determination  of  j ob-rela tedness ,  however,  must  be  based  on  a  Job 
analysis  for  the  particular  job  in  question.  It  is  highly  unlikely  that  one 
overall  list  of  competencies  could  be  used  for  all  Jobs  (Pelissero,  1984). 
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Appraisal  System  Component 


Performance  Appraisal  Under: 


CSRA 


OER 


Critical  elements  that  are  Job-  Required  Not  Required3 

specific  and  objectively  defined 
at  the  beginning  of  the  rating 
period 

Written,  objective,  Job-related  Required  Not  Required 

standards,  defined  at  the  begin¬ 
ning  of  the  rating  period, 
against  which  each  rating  is 
made . 


Prees tablished  rating  distribution 


Not  Permitted  Required13 


Non-job-specific  elements 


Not  Permitted  Required0 


I 

3  An  element  is  critical  if  it  is  "a  component  of  an  employee's  job  that  is 
of  sufficient  importance  that  performance  below  the  minimum  standard 
established  by  management  requires  remedial  action  and  denial  of  a 
wlthin-grade  increase .. .Such  action  may  be  taken  without  regard  to  other 
components  of  the  job"  (Office  of  Personnel  Management,  1979,  p.  45590).  ■ 

Under  the  OER  system,  although  job-specific  elements  may  be  defined  at  ■ 

the  beginning  of  the  rating  period,  determination  of  criticality  is  not 
required . 

k  The  senior  rater  is  required  to  assume  a  bell-shaped  normal  distribution 
for  officer  potential  and  then  compare  the  rated  officer's  potential  with 
all  other  officers  of  the  6ame  grade  rated  by  the  senior  rater 
(Department  of  the  Army,  1985). 


c  Regardless  of  grade,  position,  branch,  and  specialty,  all  Army  officers 
are  rated  on  the  competencies  and  attributes  presented  in  Table  2. 
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Professional  Competence 


o  Possess  capacity  to  acquire  knowledge/grasp  concepts 
o  Demonstrates  appropriate  knowledge  and  expertise  in  assigned  tasks 
o  Maintains  appropriate  level  of  physical  fitness 
o  Performs  under  physical  and  mental  stress 
o  Encourages  candor  and  frankness  in  subordinates 
o  Clear  and  concise  in  written  communication 


o  Displays  sound  judgment 
o  Seeks  self-improvement 
o  Is  adaptable  to  changing  situations 
o  Sets  and  enforces  high  standards 
o  Possesses  military  bearing  and  appearance 
o  Supports  EO/EEO 

o  Clear  and  concise  in  oral  communication 
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Performance  Appraisal  Needs  to  be  Viewed  as  Important  and  Worthwhile  by  Users 


In  order  for  the  PA  process  to  be  used  with  more  than  minimal,  begrudging 
compliance,  it  needs  to  be  viewed  as  Important  and  worthwhile  by  the  users. 
Yet  the  very  opposite  exists  in  many  organizations.  The  PA  process  is 
frequently  viewed  with  great  pessimism.  It  is  seen  as  an  unpleasant  burden 
and  its  usefulness  is  questioned  (see  Table  3).  Past  PA  systems  often  are 
seen  as  failures  and  little  optimism  for  the  success  of  future  systems  exists 
(see  Table  4).  These  negative  views  toward  PA  may,  however,  be  partly 
responsible  for  the  failure  of  these  systems.  Lack  of  user  support  for  PA 
undermines  the  system  (Abbott  &  Schuster,  1984;  Barrett,  1966;  Hall,  1983; 
Kirkpatrick,  1984). 

An  example  of  the  Impact  of  lack  of  user  enthusiasm  on  the  Implementation 
of  a  PA  system  occurred  in  one  Federal  agency  (Gaertner,  Gaertner,  &  Akinnus, 
1984;  for  other  examples  6ee  Conference  Board,  1979,  and  Hall,  1983).  Top 
administrative  officers  were  highly  committed  to  the  use  of  PA,  lobbied 
extensively  for  the  passage  of  CSRA,  and  upon  passage  of  the  act,  volunteered 
to  put  Merit  Pay  in  place  a  year  ahead  of  the  mandated  implementation  date. 
However,  the  agency's  operative  level  personnel  resisted  and  the  major 
program  offices  successfully  argued  that  they  be  excused  from  early 
implementation  of  the  act.  Gaertner  et  al .  (1984)  attributed  the  resistance 
by  operating-level  personnel  to  the:  (a)  lack  of  confidence  in  the  chain  of 
command,  (b)  distrust  of  agency  management,  (c)  political  atmosphere  within 
the  agency,  and  (e)  fear  that  the  innovations  would  reduce  employee  autonomy. 

Unions  also  may  resist  PA  systems  and  seek  to  ensure  that  their 
interests  are  protected.  They  traditionally  view  seniority  (i.e.,  length 
of  employment)  rather  than  PA,  as  the  basis  on  which  benefits  should  be 
allocated  and  administrative  decisions  about  promotion,  layoffs,  and  wage 
increases  should  be  made  (Gordon  &  Johnson,  1982;  Ingster,  1984).  The 
Federal  Labor  Relations  Authority  (FLRA)  has  had  to  resolve  some  performance 
appraisal  system  disputes  between  unions  and  Federal  agencies  regarding  the 
rights  of  each.  (Examples  of  decisions  on  which  aspects  of  Federal  agency  PA 
systems  are  negotiable  between  unions  and  management  and  which  are  outside 
the  duty  to  bargain  because  they  are  a  violation  of  management  right  to 
direct  employees  and  assign  work  can  be  found  in  United  States  Code  Service, 
1983,  pp.  210,  237.) 

The  subject-matter  experts  and  the  PA  literature  both  strongly  suggested 
that,  when  revising  a  PA  system,  it  is  important  to  involve  the  unions  early. 
There  should  be  a  continuous  flow  of  information  to  the  union  about  the 
appraisal  plan  (Ingster,  1984).  By  involving  unions  during  the  plan's 
development,  management  can  gain  insight  into  the  areas  which  are  of 
particular  concern  to  the  unions  and  may  be  better  able  to  design  a  plan  that 
sult6  management  objectives  and  at  the  same  time  is  acceptable  to  the  union. 
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Table  3 


Lack  of  Enthusiasm  for  Performance  Appraisal 


Performance  appraisals  are  "often  viewed  with  the  same  enthusiasm  as  Income 
tax  forms"  (McCall  &  DeVries,  1977,  p.  1). 

"It  seems  as  if  formal  performance  appraisal  is  not  so  much  an  accepted  part 
of  the  management  scene  as  it  is  a  tolera  ted  one,  something  of  a  necessary 
evil .. .However  necessary  some  formal  appraisal  system  appears  to  be,  current 
systems  are  6tlll  widely  regarded  as  a  nuisance  at  best  and  a  dangerous  evil 
at  worst"  (Conference  Board,  1979,  p.  1). 

"Performance  appraisal  systems  are  a  lot  like  seat  belts.  Most  people 
believe  they  are  necessary,  but  they  don’t  like  to  use  them"  (Latham  & 
Wexley,  1981,  p.  2). 


"The  chief  value  thu6  far  of  performance  evaluation  to  personnel  management 
seems  to  be  that  It  lends  itself  readily  to  theoretical  rambling  by  writers, 
statisticians  and  psychologists;  its  lot  in  the  shop,  office  and  warehouse 
has  been  querulous — a  burden  to  supervisors,  a  frustration  to  personnel 
administrators,  and  a  mystery  to  employees"  (Dailey,  1961,  p.  42). 


“For  the  personnel  practitioner,  the  use  of  the  present  PA  system  may 
represent  the  path  toward  'utter  despair  and  hopelessness',  while  it6 
abandonment  represents  the  path  toward  'extinction'"  (Bernardin  &  Beatty, 
1984,  paraphrasing  Woody  Allen,  p.  2). 


"Many  appraisal  systems  are  being  severely  criticized  from  all  sides. 
Managers  find  them  troublesome,  particularly  when  they  have  to  criticize  an 
employee  personally  and  put  the  criticism  in  writing.  Employees  charge  that 
the  appraisals  are  often  too  subjective,  and  the  federal  courts  frequently 
agree  with  them.  Most  disappointing  of  all,  many  executives  themselves 
realize  that  existing  performance  appraisal  systems  do  not  bring  about  a 
positive  change  in  their  employees'  behavior"  (Latham  &  Wexley,  1981, 

P-  1-2). 


Table  4 


Performance  Appraisal  Systems  Seep  as  Failures 


“The  Inaccuracy  of  the  performance  measurement  system  has  led  some  authors  to 
conclude .. .performance  appraisal[s]  are  a  waste  of  effort.  Other  authors 
have  blamed  the  failure  of  appraisal  systems  on  the  way  appraisals  are 
Implemented"  (Edwards,  1983,  p.  13). 

“There  Is  abundant  evidence  that  performance  appraisal  programs  often  fail  to 
produce  the  expected  benefits  of  opening  up  lines  of  communications  and 
enhancing  work  effectiveness,  in  addition  to  providing  administratively 
useful  appraisals"  (Finn  &  Fontaine,  1984,  p.  335). 

“Most  organizations  are  dissatisfied  with  their  performance  appraisal  (PA) 
process,  particularly  for  administrative/managerial  positions.  They  have 
concerns  about  its  objectivity,  its  relevance  and  its  validity.  In  many 
cases  the  complaint  is  that  the  appraisal  system  simply  does  not  work" 
(Schneier  &  Beatty,  1979,  p.  65). 

(Performance  appraisal]  "is  in  a  troubled  state — it  makes  great  theoretical 
sense,  but  falters  in  actual  practice"  (DeVries  et  al . ,  1980,  p.  11). 


“Performance  appraisal  has  created  more  consternation  than  perhaps  any  other 
single  issue  in  American  management.  Organizational  approaches  to 
performance  appraisal  typically  range  from  dismal  to  mediocre,  with  a  very 
few  verging  on  the  excellent"  (Albrecht,  1978,  p.  153). 

“In  general,  the  empirical  work  on  appraisal  could  be  entitled  'Another  Tning 
That  Can  Go  Wrong  with  Performance  Appraisal,  Part  1 . . . Inf ini ty ' ”  (Bernardin 
&  Beatty,  1984,  p.  2). 

“No  management  tool  has  been  given  more  lip  service  and  used  less  effectively 
than  employee  performance  appraisals"  (Beacham,  1979,  p.  53). 

“Performance  appraisal  systems  do  not  and  cannot  possibly  work... The  emerging 
problems  of  this  new  reform  (CSRA),  are  not  traceable  to  supervisory 
carelessness,  nor  to  the  inevitable  'shakedown  period'  which  follows  any 
major  change,  but  are  inherently  a  part  of  any  appraisal  system.  If,  as  it 
i6  argued,  a  given  task  is  impossible  to  perform,  no  level  of  effort  will 
overcome  that  inherent  impossibility"  (Thayer,  1981,  p.  21). 

“Performance  appraisal  has  remained  an  unsolved  and  perhaps  unsolvable 
problem  in  human-resource  management. . .To  be  sure,  many  gimmicky  innovations 
have  been  devised  for  reviewing  managerial  and  employee  performance,  yet  they 
have  proven  to  be  short-lived.  The  search  for  the  Holy  Grail  in  performance 
appraisal  goes  on"  (Patten,  1982,  p.  1). 


“In  spite  of  the  efforts  of  researchers  and  practitioners,  performance 
appraisal  systems  remain  more  of  an  albatross  than  an  effective 
organizational  tool"  (McCall  &  DeVries,  1977,  p.  11). _ 
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Top  Management  Backlog  Is  Essential 


The  importance  of  obtaining  top  management  backing  was  asserted  in  the 
literature  (e.g.,  DeVries,  et  al . ,  1980;  Henderson,  1984;  Steers  &  Lee,  1983; 
Steinberg  &  Burke,  in  press)  and  also  emphasized  by  the  subject-matter 
experts.  DeVries  et  al.  (1980,  p.  107)  concluded  tnat  "lack  of  top 
management  suppor t. .. represen ts  one  of  the  most  frequently  cited  reasons 
for  failure  of  PA  systems."  The  subject-matter  experts  confirmed  that 
obtaining  top  management  backing  should  be  the  first  item  on  the  agenda  prior 
to  designing  or  revising  a  PA  system  and  they  cautioned  that  patience  is 
required  in  that  effort.  Though  it  might  take  up  to  3  to  5  months  of  careful 
marketing  in  order  to  obtain  this  top-level  backing,  the  end  result  would  be 
worth  it. 

The  reason  for  this  strong  emphasis  on  the  role  of  top  management  i6  the 
belief  that  this  group  can  be  Instrumental  in  making  PA  work  and  in  reversing 
user  apathy  and  resistance.  Top  management  should  set  PA  policy  and  reward 
those  who  implement  it.  Top  management  should  assign  responsibility  for  PA 
to  a  credible  and  visible  key  executive,  allocate  the  resources  necessary 
to  implement  the  system,  Identify  organizational  goals  for  linkage  to 
appraisals,  and  sponsor  a  campaign  to  gain  acceptance  for  the  appraisal 
system  from  employees  at  all  levels  in  the  organization.  These  same 
Initiatives  begun  without  top  management  support  lack  the  necessary  clout 
and  accordingly  are  not  taken  as  seriously. 

The  Performance  Appraisal  Process  Should  be  Used  as  a  Management  Tool 

Both  the  literature  and  the  subject-matter  experts  advocated  that,  in 
addition  to  its  use  in  making  personnel  decisions  such  as  promotions  and 
awards,  PA  should  be  used  as  a  tool  for  management  to  plan  and  review 
performance,  motivate  employees  and  communicate  job  requirements  and 
evaluation  criteria.  This  function  of  PA  is  clearly  mandated  by  the  CSRA 
requirement  to  develop  yearly  individual  performance  plans  which  communicate 
elements  of  the  job  and  performance  standards. 

However,  these  sources  al60  revealed  that  the  PA  process  used  as  a  means 
to  improve  individual  performance  is  infrequently  practiced.  More  often  than 
not,  performance  appraisal  is  seen  as  a  personnel  department  requirement  and 
an  obstacle  that  takes  the  manager  away  from  the  important  duties  of  the  job 
(Godwin  &  Needham,  1981).  Managers  argue  that  PA: 

o  requires  too  much  time  (DeVries  et  al.,  1980). 

o  does  not  help  get  the  Job  done  (Godwin  &  Needham,  1981). 

o  is  not  a  good  tool  because  it  constitutes  delayed  feedback  and,  a6 
such,  is  not  in  accord  with  accepted  learning  principles  (Dailey  & 
Madsen,  1980). 
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o  serves  too  many  purposes.  This  results  In  feedback  which  is 
designed  to  meet  the  requirements  of  personnel  decisions,  not 
improvement  of  performance  (McMillan  &  Doyel ,  1980;  Yager,  1981). 

o  Is  uncomfortable  for  both  the  supervisor  and  subordinate  (Cohen  & 
Jaffee,  1982;  Levinson,  1976). 

One  of  the  reasons  for  these  objections  is  the  view  that  PA  is  a 
once-a-year  act  of  filling  out  forms  and  confronting  subordinates.  Advocates 
of  the  use  of  PA  as  a  management  tool,  however,  suggest  that  this  should  not 
be  so.  Rather,  they  envision  PA  as  part  of  an  ongoing  management  process 
(Baird,  et  al . ,  1982;  Conference  Board,  1979;  Meyer,  Kay,  6  French,  1965; 
Yager,  1981).  This  process  is  diagrammed  in  Table  5. 

Identification  of  objectives  at  the  beginning  of  the  appraisal  period 
(i.e.,  goal  setting,  see  Step  1  in  Table  5)  is  an  integral  part  of  developing 
the  performance  plan.  Setting  specific  goals  has  been  shown  to  improve 
performance.  (See  Latham  and  Yukl ,  1975,  for  a  review  of  goal  setting  in  27 
organizations,  and  Locke,  Shaw,  Saari,  and  Latham,  1981,  for  a  review  of  the 
goal-setting  literature  from  1969  to  1980.)  The  goal-setting  Interview, 
however,  requires  preparation  on  the  part  of  both  the  supervisor  and  the 
subordinate.  A  useful  guide  for  both  is  provided  by  Labovitz  and  Baird 
(1982). 

The  supervisory  role  Includes  coaching,  monitoring  and  documenting 
performance,  and  providing  feedback  (see  Step  2,  Table  5).  It  is  especially 
important  to  provide  continuous  feedback  throughout  the  appraisal  period,  and 
not  only  at  the  end,  because  it: 

o  provides  information  on  goal  accomplishment  and  demonstrates  concern 
about  performance  (Ilgen,  Fisher,  &  Taylor,  1979). 

o  i s  less  effective  when  delayed  (Dailey  &  Madsen,  1980;  Meyer  et  al . , 
1965). 

o  improves  the  likelihood  of  employee  acceptance  because  supervisory 
suggestions  for  improvement  do  not  overwhelm  the  subordinate  by 
being  given  all  at  once  (Meyer  et  al.,  1965). 

o  makes  le6S  likely  an  unexpected,  emotional  confrontation  at  PA  time 
(Henderson,  1984;  Yager,  1981). 

Based  on  a  review  of  16  studies,  Landy,  Farr,  and  Jacobs  (1982)  concluded 
that  feedback  improves  performance.  Earlier,  Landy,  Barnes,  and  Murphy 
(1978)  found  that  identification  of  goals  and  frequency  of  evaluation  are 
both  related  to  the  perceived  fairness  and  accuracy  of  performance 
evaluation.  A  note  of  caution,  however,  is  in  order.  Feedback  doe6  not 
have  uniformly  successful  results  (Ilgen  et  al . ,  1979;  Meyer,  1977).  For 
some,  feedback  may  fail  to  improve  performance  and  even  be  detrimental  to 
it.  Very  frequent  feedback  from  supervisors,  for  example,  may  result  in 


Table  5 

Steps  of  the  Performance  Appraisal  Process 


S  tep  1 


Supervisor  and  subordinate  develop  written  performance  plan  at  the 
beginning  of  the  appraisal  period.  The  plan  reflects  organizational 
goals  and  Job  requirements.  It  contains  critical  elements  and 
objective  standards  which  will  be  the  basis  for  the  appraisal  at 
the  end  of  the  year. 


Step  2 


Supervisor  monitors  and  documents  performance  of  subordinate, 
coaches  subordinate,  and  provides  feedback  on  performance. 

Subordinate  monitors  and  documents  own  accompl i shmen ts  and 
problems  encountered. 


Step  3 


Supervisor  conducts  an  interim  review — supervisor  and  subordinate 
compare  performance  on  Job  elements  to  performance  standards, 
plan  to  Improve  performance  if  necessary,  and  revise  performance  plan 
if  necessary  to  take  into  account  changing  requirements  or  conditions. 


S tep  4 

Repeat  step  2. 

Step  5 


Repeat  step  3,  if  needed. 


•  \ 

U 


Step  6  ' 

—  m 


Supervisor  makes  appraisal  and  conducts  the  PA  interview. 


the  perception  of  loss  of  control  and  this,  in  turn,  may  lead  to  excessive 
reliance  on  others.  The  perception  of  the  feedback  and  the  response  to  it 
are  a  function  of  the  source  of  the  feedback,  the  message,  and  the  recipient. 
(For  a  thought-provoking  review  on  this  literature,  see  Ilgen  et  al  . ,  1979.) 
Issues  related  to  successful  feedback  should  be  dealt  with  extensively  in 
managerial  training  programs. 

Self-monitoring  and  documenting  of  performance  by  the  subordinate  should 
be  encouraged  by  the  supervisor.  The  subordinate  thereby  becomes  part  of  the 
process  Instead  of  a  bystander  (Ilgen  et  al.,  1979).  Also,  self-monitoring 
of  results  of  past  performance  has  been  shown  to  improve  future  performance 
(Landy  et  al . ,  1982)  . 

The  interim  review  (Step  3,  Table  5)  serves  several  functions:  it  tells 
employees  how  they  are  doing;  it  provides  the  opportunity  to  plan  for 
improvement;  and  it  allows  reassessment  of  desired  goals  (Conference  Board, 
1979).  Organizational  goals  and  expectations  do  change  and  it  is  Important 
to  communicate  and  clarify  these  mission  changes  (Bates,  1984),  and  update 
the  performance  plan.  However,  experience  shows  that  when  the  procedure  to 
modify  the  performance  plan  mid-cycle  is  too  much  trouble  (i.e.,  requires 
considerable  paperwork  or  second-level  approval),  the  plan  Is  simply  ignored. 
Some  sort  of  addendum  attached  to  the  performance  plan  may  be  one  easy 
solution . 


The  linkage  of  PA  to  organizational  goals  is  highly  recommended. 

It  is  mandated  by  recent  law  (see  above)  and  was  advocated  by  both  the 
6ubject-matter  experts  and  the  literature  (DeVries  et  al . ,  1980;  Henderson, 
1982;  Odiorne,  1979).  Though  the  concept  appears  to  be  fairly 
straightforward,  putting  it  into  practice  is  not.  It  requires  that 
(1)  organizational  goals  be  identified  by  top  management  and  communicated 
throughout  the  organization,  and  that  (2)  PA  be  linked  to  the  accomplishment 
of  these  goals. 

For  PA  purposes,  mission  statements  which  identify  and  describe  the 
reason  for  the  existence  of  the  organization  are  often  much  too  general. 

They  need  to  be  stated  in  terms  of  specific  organizational  objectives  and 
updated  periodically  so  that  work  activities  throughout  the  organization  may 
be  directed  toward  the  same  objectives  and  coordinated.  The  process  of 
Identifying  objectives  should  start  at  the  top  of  the  organization  and  filter 
down  through  the  organizational  hierarchy  (Henderson,  1982;  Lee  6  Zwerman, 
1976;  Odiorne,  1979).  At  each  level,  they  need  to  be  refined  further  into 
intermediate  objectives  and  into  the  tasks  required  to  accomplish  them.  This 
process  has  aptly  been  called  "cascade  of  goals”  (Raia,  1974).  It  is 
designed  to  assure  that  work-unit  goals  interlock  and  support  organizational 
objectives  and  to  allow  the  translation  of  organizational  planning  into 
specific  individual  objectives  and  performance  plans. 

The  identification  of  organizational  goals  and  the  linkage  of  PA  to  them 
is  an  integral  part  of  management  by  objectives  (MBO),  as  originally  proposed 
by  Peter  Drucxer  (1954).  According  to  Drucker,  a  manager's  performance 
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should  be  evaluated  by  Its  contribution  to  accomplishment  of  organizational 
goals.  He  recommended  that  managers  require  subordinates  to  write  a 
"manager's  letter"  twice  a  year.  In  it,  subordinates  should:  (1)  define  the 
objectives  of  their  own  jobs  and  that  of  their  supervisor,  (2)  write  their 
performance  standards,  (3)  identify  major  obstacles  to  the  accomplishment  of 
objectives,  and  (4)  indicate  proposed  activities  to  accomplish  the  objectives 
(Abbott  &  Schuster,  1984). 

The  basic  MBO  approach  has  been  Incorporated  into  CSRA  and  the 
performance  contract  1b  embodied  in  the  CSRA-required  performance  plan. 

It  must  contain  the  critical  elements,  and  non-critical  elements  if  any, 
and  performance  standards  for  the  Job.  Until  the  passage  of  the  Civil 
Service  Retirement  Spouse  Equity  Act  of  1984,  there  was  no  requirement  to 
show  how  accomplishments  were  related  to  organizational  goals.  The  law  does 
not,  require,  however,  that  there  be  any  systematic  coordination  of 
performance  plans  (such  as  the  "cascade-of-goal6”  approach)  to  assure  that 
goals  are  addressed  in  an  integrated  fashion. 

An  example  of  the  MBO  process  in  action  in  the  Federal  government  can  be 
found  in  the  Department  of  Commerce.  A  letter  from  the  Secretary  of  Commerce 
to  all  employees  of  the  agency  is  provided  in  Appendix  A.  This  letter 
communicates  agency  goals  for  FY  1985-1988  and  full  top  management  support 
for  the  MBO  planning  and  management  system  used  to  accomplish  these  goals. 

The  goals  (provided  in  Appendix  A,  as  well)  are  general  enough  to  cut  across 
organizational  lines  and  yet  specific  enough  for  their  accomplishment  to  be 
verified.  A  computerized  system  exists  to  track  the  "cascade  of  goals"  and 
identify  individual  accountability  for  their  attainment. 

The  Performance  Appraisal  System  Should  be  Adapted  to  Organizational  Needs 

The  interviewees  frequently  expressed  the  need  to  adapt  their  PA  systems 
to  special  features  and/or  missions  of  their  agencies.  Adaptations  in 
response  to  particular  missions  have  involved  the  type  of  Job  analysis  and  by 
whom  conducted,  degree  of  uniformity  of  elements  and  standards,  timing  of 
appraisals,  and  type  and  amount  of  training.  Specific  conditions  mentioned 
which  might  call  for  adaptations  were  specialized  occupations,  many  (or  few) 
individuals  in  a  given  job  series,  amount  of  geographical  dispersion  of  the 
organization,  many  small  units  working  in  isolation  from  the  rest  of  the 
organization,  organizational  size  (large  or  small),  and  cyclical  calendar 
events  which  Impact  on  scheduling  of  performance  appraisals. 

Although  legal  constraints  prescribe  the  main  features  of  PA  6y6tems 
under  CSRA  (see  above),  each  organization  has  much  flexibility  of 
implementation  in  areas  such  as  training  (how  people  are  trained  and  by  whom, 
content  and  frequency  of  training).  Job  analysis  (how  it  is  done  and  by  whom, 
development  of  generic  elements  and/or  standards),  and  linkage  to 
organizational  goals  (how  the  goals  are  determined,  communicated,  and  linked 
to  PA).  Since  it  is  recommended  that  PA  be  considered  a  part  of  the 
management  process,  it  is  logical  that  managers,  who  will  be  the  users  and 
promoters  of  the  system,  should  be  involved  in  its  development  and  adaptation 
to  organizational  needs. 
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V. 


System  Perpetuation  Should  Be  Planned 


Another  theme  addressed  by  the  literature  and  the  subject-matter  experts 
was  the  need  to  place  more  emphasis  on  mechanisms  which  maintain  the  momentum 
of  newly  initiated  PA  systems.  Often  after  the  initial  introduction  to  the 
system,  managers  hear  no  more  about  it  from  their  supervisors,  subordinates, 
personnel  department,  and  higher  management  (DeVries  et  al.,  1980).  With  no 
ongoing  reminders,  support,  and  encouragement,  managers  are  not  likely  to  use 
the  PA  process  oh  a  continuing  basis. 

It  i6  not  surprising,  therefore,  that  PA  systems  traditionally  have  had 
very  brief  shelf  lives.  The  initial  enthusiasm  for  new  systems  soon  turns 
into  disillusionment  and  the  system  is  discarded  shortly  in  favor  of  a  new 
one  that  supposedly  will  improve  on  the  previous  one  (Conference  Board, 

1979).  Some  examples  follow: 

o  The  first  formal  PA  system  in  the  U.S.  Federal  government  was 

Introduced  in  1842  and  died  of  half-hearted  practice  and  apathy  by 
the  end  of  the  1850s  (Lopez,  1968).  The  next  Federal  PA  system 
wasn't  begun  until  1887. 

o  The  U.S.  Navy  used  48  different  fitness  reports  in  the  91  years  from 
1865  to  1956  (Lopez,  1968). 

o  In  a  survey  of  almost  300  industrial  firms,  more  than  half  reported 
that  their  current  PA  systems  had  been  developed  within  the  last  3 
years  (Conference  Board,  1979). 

o  In  a  survey  of  18  major  national  and/or  International  corporations 
In  the  private  sector,  Teel  (1980)  found  that  6  had  systems  which 
were  in  operation  for  less  than  a  year  and  9  were  currently  being 
revised.  Only  5  had  systems  that  were  in  use  over  5  years. 

Additional  reasons  cited  for  the  abandonment  of  PA  systems  include: 

(a)  changing  legal  requirements  (b)  user  resistance,  and/or  (c)  inflation  of 
ratings.  User  resistance  may  be  related  to  the  process  of  PA  in  general,  or 
to  some  specific  feature  of  a  system  such  as  the  rating  format.  Inflation  cf 
ratings,  in  which  most  people  are  rated  at  or  near  the  top  of  the  scale,  most 
probably  reflects  something  other  than  the  existence  of  superior  groups  of 
employees  (Bernardin,  Orban,  &  Carlyle,  1981;  Ilgen  &  Feldman,  1983).  It  may 
reflect  (a)  the  perception  that  the  other  raters  in  the  organization  base 
ratings  on  these  factors  and  that  they  must  also  if  their  subordinates  are  to 
remain  competitive  for  promotions  and  awards;  (b)  the  feelings  of  the  rater 
about  performance  ratings  in  general;  (c)  the  rater'6  ability  to  rate 
accurately;  and  (d)  the  concern  of  the  rater  about  the  subord ina te ' 6  response 
and  future  cooperation. 

It  has  been  suggested  that,  in  order  to  maintain  interest  in  PA  systems 
beyond  the  initial  period  of  introduction,  organizations  should  discard  the 
mechanistic  notion  that  such  systems  can  be  Installed  like  pieces  of  new 
hardware  which  continue  to  operate  by  themselves  (Albrecht,  1978).  At  the 
very  least,  a  con tinued  effort  should  be  made:  (a)  to  address  the  concerns 


of  supervisors  and  subordinates  about  how  the  system  will  be  and  is  being 
used;  (b)  to  conduct  training  to  develop  skills  in  the  use  of  the  system;  and 
(c)  to  communicate  top-level  support  of  the  system  (Barrett,  1966).  One  way 
to  assure  that  this  continued  effort  is  made  might  be  to  appoint  key  line 
managers  (and/or  core  groups,  the  number  and  level  of  which  would  depend  on 
the  structure  of  the  organization)  to  be  responsible  for  the  PA  system  and 
it6  operation  as  part  of  the  management  process. 

Training  is  Essential 

Training  In  PA  is  recommended  in  the  literature  for  a  variety  of 
purposes.  It  can  be  used  to  teach  managers:  to  observe  behavior  accurately; 
to  reduce  rating  errors  (halo,  leniency,  contrast  effects,  slmilar-to-me 
errors,  central  tendency,  recency,  latency,  and  6tereotypy);  to  conduct 
better  appraisal  discussions;  to  improve  supervisory  skills  in  coaching;  to 
Increase  knowledge  of  the  performance  appraisal  system;  and  to  improve 
attitudes  toward,  and  motivation  to  use,  PA  systems  (Davis  &  Mount,  1984; 
Decker,  1982;  Dunnette  &  Borman,  1979;  Glasgow,  Simkins,  &  Guerrieri,  1981; 
McIntyre,  Smith,  &  Hassett,  1984;  Pursell,  Dossett,  &  Latham,  1980;  Spool, 
1978;  Zedeck  &  Cascio,  1982).  Formal  training  techniques  reported  in  the 
research  literature  include  various  combinations  of:  reading  assignments, 
lectures  (brief  and  long),  discussions,  diary  keeping,  videotapes,  rating 
practice,  feedback,  self-paced  Instruction,  computer-assisted  instruction, 
role  playing,  and  behavior  modeling.  Topics  that  have  been  recommended  for 
inclusion  in  PA  training  programs  are  listed  in  Table  6.  A6  can  be  seen 
there,  some  topics  primarily  require  learning  facts  whereas  others  require 
learning  skills  as  well.  Unfortunately,  the  literature  on  teaching  methods 
does  not  distinguish  the  methods  that  work  best  for  each  purpose. 

Much  research  has  been  conducted  on  the  effectiveness  of  training  as 
a  means  to  reduce  rater  error.  For  example,  Dunnette  and  Borman  (1979) 
concluded  that  errors  can  be  reduced  via  training,  and  Spool  (1978),  in  a 
review  of  25  years  of  research  literature  on  training,  concluded  that  in 
almost  all  of  the  studies  he  reviewed,  training  improved  accuracy  in 
observing  behavior.  Yet  there  is  other  evidence  that  training  does  not 
always  reduce  rater  error.  Davis  and  Mount  (1984)  found  that  training 
managers  did  not  affect  halo  or  leniency  tendencies,  and  Warmke  and  Billings 
(1979)  found  that  although  training  reduced  errors  on  experimental  ratings, 
it  had  no  effect  on  subsequent  administrative  ratings.  What  the  Davis  and 
Mount,  and  Warmke  and  Billings  studies  had  in  common  were  ratings  which  were 
for  actual  use  in  the  organization.  The  influence  of  purpose  of  rating  on 
determining  success  of  training  to  reduce  rater  error  is  illustrated  in  the 
research  of  Zedeck  and  Cascio  (1982).  These  researchers  manipulated  the 
purpose  of  rating  (either  to  recommend  development,  to  award  a  merit  raise, 
or  to  retain  a  probationary  employee)  and  found  that  there  was  le6S 
discrimination  among  ratees  when  the  rating  was  for  the  purpose  of  deciding  a 
merit  increase. 


The  findings  that  differential  effects  of  training  depend  on  purposeof 
the  training  also  calls  into  question  the  usefulness  of  the  many  studies  of 
training  using  college  students  making  ratings  for  research  purposes.  The 
findings  of  these  studies  may  be  far  less  predictive  of  the  results  of 


training  In  working  organizations  than  studies  in  which  the  subjects  are 
actual  supervisors  making  ratings  for  administrative  purposes  (Wannke  & 
Billings,  1979).  In  similar  fashion,  research  based  upon  what  Ilgen  and 
Favero  (1985)  call  the  "paper  people  paradigm"  (in  which  raters  are  presented 
with  written  information  about  a  stimulus  person)  may  not  necessarily 
generalize  to  the  way  people  respond  to  actual  people.  This  paradigm  does 
not  take  into  account  PA  as  a  complex,  cyclical  process  which  involves  the 
selection,  integration,  and  recall  of  information  over  time  (Feldman,  1981). 

It  also  does  not  take  into  account  the  social,  economic,  and  professional 
consequences  of  ratings  and  resultant  personnel  actions  that  must  be  lived 
with  and  accommodated  to  by  both  the  rater  and  ra tee  (McCall  &  DeVries, 

1977). 

Another  factor  to  consider  with  respect  to  training  to  reduce  rater 
error  i6  the  criteria  for  determining  the  amount  of  rater  error.  Many 
studies  have  found  that  training  raters  reduces  rater  error  (Landy  &  Farr, 
1983).  However,  in  most  of  these  studies,  the  criteria  for  determining  rater 
error  involved  inspection  of  the  response  distribution  as  opposed  to 
comparison  of  the  ratings  to  true  criterion  scores.  The  few  studies  using 
the  latter  approach  indicate  that  training  does  not  improve  rater  accuracy, 
and  even  may  decrease  it  (Bernardin,  1979;  Bernardin  &  Pence,  1980). 

Other  research  with  respect  to  training  has  shown  that  training  can 
definitely  increase  knowledge  of  the  PA  system  and  can  improve  skills  in 
formulating  developmental  plans  for  subordinates  (Davis  &  Mount,  1984; 

Glasgow,  Simkins,  &  Guerrieri,  1981).  Interviewees  from  several  Federal 
agencies  indicated  that  training  for  appraisal  purposes  and  for  performance 
management,  in  general,  was  needed  to  make  appraisal  systems  work,  but  that 
lack  of  resources  for  training  was  a  problem.  Several  agencies,  however, 
were  able  to  commission  consultants  to  design  training  materials  (manuals  and 
videotapes)  for  them. 

One  approach  to  developing  training  materials  for  PA  is  the  modular 
approach  (Bianco,  1984).  The  modular  concept  offers  the  advantages  of 
allowing  the  combination,  deletion,  or  rearrangement  of  portions  of  the 
training  program  to  meet  organizational  needs.  With  the  modular  approach, 
programs  can  consist  of  core  modules  that  apply  agency-wide.  These  can  be 
supplemented  with  additional  modules  specific  to  unit/departmental 
requirements,  the  particular  audience  (e.g.,  supervisor  or  non-supervisor), 
and  the  purpose  of  training  (introductory/orientation,  in-depth,  or 
refresher) . 

One  training  program  of  special  interest,  because  it  illustrates  an 
application  of  several  themes  discussed  above,  is  that  of  the  U.S.  Army 
Materiel  Command  (AMC).  AMC  arranged  with  OPM  to  design  and  conduct  a 
training  workshop,  specifically  tailored  to  Army  requirements,  on  teaching 
supervisors  to  develop  performance  standards  and  individual  performance  plans. 
Top  management  clearly  communicated  its  strong  backing  for  this  training. 

The  commanding  General  requested  AMC  commanders  to  recommend  two  candidates 
each  to  receive  the  training  and  then  conduct  supervisory  training  at  the 
installation  level  (see  Appendix  B).  One  candidate  was  to  be  from  the 
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Table  6 


Recommended  Performance  Appraisal  Training  Topics 


To  Provide  Information 


To  Improve  Skills 


Definition  of  terms 
(PA  cycle,  elements, 
critical  elements, 
performance  standards, 
performance  plans,  etc.) 


Developing  performance  plans 
(identifying  critical 
elements,  linking  employee 
objectives  of  organizational 
goals,  developing  objective 
standards,  etc.) 


Purposes  of  performance 
appraisal 


Documenting  performance 

(diary,  critical  incidents, 
etc . ) 


Legal  requirements 


Conducting  informal  day-to-day 
coaching  and  feedback 


Organizational  requirements 
(regulations,  forms,  due 
dates,  etc.) 


Reducing  rater  error 


Preparing  for  and  conducting 
interim  reviews 


Preparing  for  and  conducting 
appraisal  interviews 


Civilian  Personnel  Office  (CPO)  staff  and  one  from  line  management.  By 
sending  managers  to  the  workshop  in  addition  to  per sonnel 1 s ts ,  top  management 
Indicated  that  managers  have  an  Important  role  In  Insuring  the  success  of  the 
PA  system.  Also,  system  perpetuation  was  fostered  by  training  supervisors  to 
be  on-slte  experts  and  giving  them  responsibility  for  training  others  to  make 
the  system  work.  To  facilitate  the  training  role,  workshop  participants  were 
provided  with  (a)  a  detailed  Instructor's  Manual  for  their  use  in  delivering 
Instruction  back  home,  including  copies  of  overhead  projector  slides,  (b)  a 
training  plan  for  conducting  the  Instructor  Briefing  sessions,  (c)  a 
camera-ready  copy  of  a  Participant's  Coursebook  to  be  used  in  conducting 
sessions  of  the  course,  and  (d)  an  evaluation  instrument  for  use  in  obtaining 
participant  feedback  on  sessions. 

Ease  of  Use  is  Important 

Many  Federal  agencies  are  searching  for  ways  to  meet  user  demands  for  PA 
systems  that  are  easier  to  apply  than  those  currently  in  use.  Of  particular 
concern  are  two  issues:  the  amount  of  paperwork  and  difficulty  involved  in 
developing  performance  plans. 

Proposed  solutions  for  reducing  the  paperwork  burden  focus  on  the  appraisal 
forms.  The  solutions  include  simplifying  the  forms  to  make  them  easier  to 
complete,  combining  them  so  that  the  same  form  can  be  U6ed  for  more  than  one 
pay  category,  making  them  word-processor  compatible  to  reduce  typing  required, 
and  permitting  attachments  to  the  forms  to  reflect  changes  In  requirements 
during  the  rating  period  rather  than  rewriting  the  entire  document  (see 
Steinberg  &  Burke,  in  press). 

The  solutions  considered  for  reducing  the  difficulty  in  developing 
performance  plans  can  be  categorized  into  one  of  two  approaches.  The  first 
provides  more  and  better  training  for  the  supervisor  to  develop  individual 
performance  plans  (e.g.,  see  the  discussion  of  the  AMC  training  course, 
above).  The  second  approach  iovolves  developing  performance  plans  (or 
portions  of  them)  that  can  be  used  for  a  specified  group  of  employees,  and 
which,  consequently,  need  not  be  developed  separately  for  each  one.  This  is 
accomplished  either  by  retaining  the  performance  plan  format  but  developing 
generic  elements  and/or  standards  which  apply  to  a  whole  group  of  employees, 
or  by  using  a  different  rating  format,  such  as  a  rating  scale,  to  communicate 
elements  and  staadard6.  The  two  alternatives  of  this  second  approach  are 
frequently  considered  because  they  relieve  the  individual  supervisor  of  much 
of  the  burden  of  developing  performance  plans.  In  addition,  the  rating-scale 
alternative  lias  received  considerable  attention  in  the  literature.  For  these 
reasons,  both  alternatives  will  be  presented  in  more  detail  below. 

Generic  Elements  and  Standards.  Since  CSRA  requires  that  the  PA  elements 
and  standards  relate  to  the  given  Job,  development  of  generic  elements  and 
standards  (i.e.,  ones  which  apply  to  a  whole  group  of  employees)  should  be 
based  on  a  Job  analysis  of  the  cluster  of  Jobs  covered.  For  large  groups  of 
employees,  this  may  require  outside  consultants  and  can  be  very  co6tly  and 
time-consuming. 
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One  set  of  generic  elements  and/or  standards  can  cover  a  few  related  job 
series  within  an  agency  (e.g.,  secretarial/clerical  positions),  or  it  may 
apply  to  a  much  larger  portion  of  the  organization  (e.g.,  all  non-PMRS, 
non-SES  employees).  One  disadvantage  of  generic  standards  is  that  a6  the 
number  and  types  of  positions  that  are  covered  by  the  same  set  of  standards 
increase,  the  standards  become  less  job-specific.  Although  the  standards 
may  still  be  appropriate,  they  would  be  less  likely  to  mention  specific 
job-related  projects  and  due  dates,  for  example.  To  address  this  i66ue,  PA 
systems  may  provide  for  generic  standards  which  can  be  modified  by  the 
addition  of  individual  work  plans,  or  permit  unique  elements  and  standards  to 
be  added  to  model  performance  plans.  In  this  way,  more  specific  expectations 
can  be  communicated  to  subordinates  and  tbe  performance  plans  can  be  modified 
to  reflect  changes  in  requirements.  It  also  means,  however,  that  more  of  the 
burden  of  developing  performance  plans  is  returned  to  the  supervisor. 

Rating  Scale  Formats.  Rating  scales  have  a  number  of  features  that  may 
make  them  seem,  at  least  on  the  surface,  even  more  desirable  than  generic 
elements  and  standards.  Since  a  scale  can  be  used  for  a  large  number  of 
employees,  it  too  removes  the  burden  of  developing  the  appraisal  instrument 
from  the  individual  supervisor.  (Scales  also  require  a  job  analysis  of 
positions  they  cover.)  In  addition,  scales  appear  simpler  to  U6e  and  easier 
to  understand  than  lengthy  performance  plans.  Al6o,  they  have  been  popular 
formats  for  PA  for  some  time,  and  continue  to  be  used  widely  (DeVries  et  al . , 
1980).  For  example,  during  the  fall  of  1981,  Tyer  (1982)  conducted  a  survey 
of  the  50  state  governments  and  found  that  rating  scales  for  PA  purposes  are 
6ti’l  popular — they  were  used  by  at  least  half  of  the  state  governments. 

There  are,  however,  6ome  disadvantages  to  using  rating  scales  in  a 
Federal  PA  system.  First,  various  features  of  PA  scales,  in  their  current 
formats,  are  Inconsistent  with  CSRA  requirements.  Second,  scales  are  costly 
and  time-consuming  to  develop.  These  two  issues  are  addressed  in  greater 
detail  below  because: 

(a)  Although  the  PA  literature  is  replete  with  descriptions  of  various 
types  of  rating  scales  and  research  conducted  on  them  (e.g., 
Bernardin  &  Beatty,  1984;  Landy  &  Farr,  1980;  and  Schwab,  Heneman, 

&  DeCotiis,  1975),  there  exists  no  systematic  review  of  their 
ability  to  meet  CSRA  requirements,  and 

(b)  Although,  many  subject-matter  experts  suggested  that  rating  scales 
may  be  easier  to  use  than  individual  performance  plans,  they  were 
not  aware  of  the  costly  and  time-consuming  process  to  develop  them. 

In  order  for  rating-scale  formats  to  meet  CSRA  specifications,  they  must 
contain  objective,  job-related  standards  (anchors)  defined  at  tbe  beginning 
of  the  rating  period,  against  which  job  performance  can  be  compared.  A 
discussion  of  the  more  popular  formats  reported  in  the  literature,  and  how 
they  stand  with  respect  to  these  criteria,  follows. 
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o  Graphic  rating  scales  are  scales,  each  of  which  represents  the  range 
of  a  given  trait  or  factor  (such  as  motivation,  cooperation,  or  job 
knowledge).  The  rater  indicates  how  much  of  the  factor  the  ratee 
has  (DeVries  et  al . ,  1980).  The  scale  may  be  a  continuous  line  or 
it  may  be  divided  into  several  discrete  categories  anchored  by 
numbers  or  a  series  of  adjectives  such  as  "low"  through  "high"  or 
"below  average"  to  "above  average"  (McCormick  &  Ilgen,  1980; 
Muchiasky,  1983;  Siegel  &  Lane,  1982).  These  anchors  do  not  provide 
the  CSRA-required  specific  Job-related  behavioral  definitions.  As 
such,  they  do  not  provide  an  objective  basis  for  the  rater  to  select 
a  given  rating,  nor  do  they  convey  to  ra tees  the  basis  upon  which 
they  will  receive  a  given  rating. 

o  Employee  comparison  formats  include  rank-order  scales,  paired- 

comparison  scales  and  forced-distribution  scales  (Muchlnsky,  1983). 
These  all  provide  for  the  rating  of  individuals  by  comparing  them 
to  each  other,  instead  of  to  a  pre-de fined  performance  standard  as 
required  by  CSRA.  Rank-order  scales  require  the  ranking  of 
employees  from  high  to  low;  paired-comparison  scales  require 
that  each  employee  be  compared  to  every  other  one;  and  forced- 
distribution  scales  require  that  the  ratings  conform  to  a  preset 
distribution. 

o  Forced-choice  scales,  introduced  by  Robert  Wherry  in  the  early  1940s 
(and  used  at  one  time  for  Army  officer  ratings,  Sisson,  1948),  are 
designed  to  prevent  rater  attempts  to  distort  ratings  in  favor  of  or 
against  a  given  individual.  They  do  this  by  masking  the  weightings 
assigned  to  each  statement  so  that  the  raters  cannot  be  certain  which 
is  more  favorable  (Bernardin  &  Beatty,  1984).  This  clearly  would  not 
be  compatible  with  the  CSRA  requirement  to  communicate  standards  at 
the  beginning  of  the  rating  period. 

o  Mixed-standard  scales  also  attempt  to  prevent  rater  manipulation  by 
disguising  the  scoring  procedure  (Rosinger,  Meyers,  Levy,  Loar , 
Mohrman,  &  Stock,  1982).  Statements  reflecting  three  degrees  of  each 
trait  or  behavior  to  be  rated  are  all  intermixed  so  that  both  the 
dimension  that  each  statement  represents  and  the  order-of-mer i t  of 
the  triad  are  not  obvious  (Bernardin  &  Beatty,  1984;  Blanz  & 

Ghiselli,  1972).  In  other  words,  the  elements  and  standards  would  be 
masked  in  contravention  of  CSRA  specifications. 

o  Behavioral  observation  scales  (BOS)  consist  of  lists  of  behavioral 
statements,  derived  from  a  critical  incident  job  analysis,  which  are 
all  rated  on  one  five-point  Likert  scale  for  frequency  with  which 
the  ratee  engages  in  the  stated  behavior  (Latham,  Fay,  &  Saarl, 

1979).  The  behavioral  statements  may  be  grouped  into  dimensions 
(i.e.,  elements,  in  CSRA  terms).  The  ratings  on  each  of  the 
behavioral  statements  are  summed  to  determine  the  overall  score 
for  each  dimension,  and  these,  in  turn  are  summed  to  determine  the 
overall  evaluation  score.  Management  determines  the  range  of  scores 
that  corresponds  to  each  level  of  the  performance  standards. 
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The  five-point  frequency  scale  on  which  the  behavioral  statements 
are  rated  may  have  two  anchors,  such  as  "almost  never"  and  "almost 
always"  at  either  end,  or  it  may  have  anchors  at  each  point  which 
indicate  the  percentage  of  time  that  the  listed  behavior  occurs  when 
required . 

Two  features  of  BOS  scales  which  may  be  inconsistent  with  CSRA 
requirements  are  presented  below: 

-  The  same  frequency  scale  is  used  for  all  task  statements. 

However,  identical  numerical  ratings  for  different  task  statements 
do  not  necessarily  represent  the  same  level  of  satisfactory 
achievement  (Bernardin  &  Kane,  1980).  Therefore  their  translation 
to  performance  standard  levels  is  inappropriate.  To  illustrate 
this  point,  consider  the  following  example.  Here  are  two  tasks 
from  a  BOS  for  an  airplane  mechanic  and  the  scale  on  which  they 
are  ra  ted : 


1  *=  0  -  642  of  the  time 

2  =  65%  -  747.  of  the  time 

3  =  75%  -  84%  of  the  time 

4  «  85%  -  94%  of  the  time 

5  =  95%  -  100%  of  the  time 

— Keeps  an  orderly  tool  box. 

— Reports  life-threatening  problems  promptly. 

It  is  conceivable  that  keeping  an  orderly  tool  box  80%  of  the 
time  might  be  considered  satisfactory,  but  reporting  airplane 
mechanical  problems  promptly  80%  of  the  time  might  not  be. 

Yet,  using  the  above  scale,  both  behaviors  would  receive  the 
identical  rating  of  "3."  Further,  when  scores  on  items  such  as 
these  are  summed,  the  score  does  not  indicate  whether  the  total 
of  "6,"  for  example,  resulted  from  a  score  of  "2"  on  the  first 
item  and  "4"  on  the  second,  or  the  other  way  around. 

-  items  which  are  observed  very  frequently  or  very  infrequently 
are  not  included  in  the  scale  because  they  do  not  differentiate 
between  good  and  poor  performers  (Latham  et  al . ,  1979). 
Therefore,  many  behaviors  which  are  required  for  the  job,  and 
even  some  that  are  critical  to  it,  do  not  enter  Into  the  rating. 

Behavlorally  anchored  rating  scales  (BARS)  originally  referred  to  as 
behavioral  expectation  scales  (BES),  have  behavioral  anchors  based 
on  critical  incidents  aud  clearly  indicate  the  weight  of  each. 
However,  the  anchors  do  not  represent  standards  which  must  be  met. 
Rather,  they  are  behaviors  which  the  raters  are  to  infer  could  be 
expected  to  occur,  given  past  ra tee  behaviors.  Raters  are  to  make 
these  inferential  leaps  from  actual  behaviors  to  expected  behaviors 
based  upon  their  own  implicit  beliefs  about  how  people  behave 
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(Nathan  &  Alexander,  1985;  Smith  &  Kendall,  1963).  Thus,  the  CSRA 
requirement  of  providing,  at  the  beginning  of  tl>e  rating  period, 
standards  against  which  Job  performance  can  be  compared,  would  not 
be  met. 

In  their  original  article  presenting  BARS,  Smith  and  Kendall  (1963) 
suggested  the  use  of  general  statements  representing  high,  low,  and 
acceptable  levels  of  performance  in  addition  to  the  scale  anchors, 
in  order  to  help  raters  determine  the  appropriate  rating.  However, 
they  expressed  concern  that  these  statements,  in  addition  to  the 
behavioral  anchors,  would  result  in  ambiguous  ratings.  Although 
some  research  supports  the  desirability  of  using  these  general 
statements  (Bernardin,  LaShells,  Smith,  &  Alvares,  1976),  they  are 
rarely  included  in  BARS.  In  fact,  standard  texts  in  the  field  of 
personnel  psychology  do  not  even  mention  them.  For  example,  although 
Casclo  (1982),  Landy  (1985),  McCormick  and  Ilgen  (1980),  Miner  and 
Miner,  (1977),  Muchinsky  (1983),  and  Siegel  and  Lane  (1982)  spend 
from  one  and  one  half  to  three  and  one  half  pages  each  on  BARS  and 
the  detailed  procedures  to  develop  them,  none  mentions  the  general 
sta  tements . 

The  fact  that  the  general  statements  of  levels  of  performance  are 
similar  to  performance  standards  required  by  CSRA  (see  Bernardin  & 
Beatty,  1984,  p.  84  and  p.  87  for  two  examples),  raises  the 
possibility  that  BARS  meet  the  requirements  of  CSRA.  However,  the 
issue  of  whether  the  "could-be-expec ted- to"  anchors  could  be  used 
still  remains. 

In  addition  to  the  deficiencies  in  meeting  CSRA  requirements  discussed 
above,  a  second  factor  should  be  taken  into  account  by  those  who  hope  to  find 
scales  an  easy  solution.  The  development,  use,  and/or  scoring  of  scales  can 
be  much  more  difficult  than  would  seem  at  first  glance.  Although  the 
following  discussion  illustrates  these  points  using  BARS  examples,  similar 
problems  are  encountered  with  other  scales  as  well. 

o  More  than  a  mere  check-mark  on  the  scale  may  be  required  of  the 

rater.  The  recording  of  Incidents  of  observed  ra tee  behavior  during 
the  rating  period  (three  or  more  incidents  for  each  of  about  5  to  10 
scales,  for  each  ratee)  is  recommended  for  BARS  (Bernardin  et  al . , 
1976;  Smith  &  Kendall,  1963). 

o  Many  people  must  be  involved  in  the  development  of  scales.  For 
BARS,  Smith  and  Kendall  (1963),  for  example,  used  542  head  nurses; 
Bernardin  (1977)  used  118  students;  and  Landy,  Farr,  Saal ,  &  Freytag 
(1976)  used  168  supervisors  to  develop  one  particular  BARS 
instrument  and  240  peers  to  develop  another. 

o  Many  critical  incidents  must  be  developed  before  the  anchor  points 
on  the  BARS  scale  can  be  set.  Goodale  and  Burke  (1975),  for 
example,  collected  360  incidents  which  they  then  narrowed  down  to  6 
or  7  for  each  of  10  dimensions. 
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o  The  scale  development  procedure  is  an  arduous,  Iterative  process 
which  is  costly  and  time-consuming  (Bernardln  &  Beatty,  1984;  Landy 
&  Farr,  1980). 


o  Because  of  the  long  and  expensive  procedure  to  develop  BARS,  the 
scales  are  not  easily  adapted  to  changing  job  requirements. 


o  Because  there  is  no  procedure  in  the  development  of  a  BARS  to  ensure 
that  the  anchors  for  a  dimension  are  mutually  exclusive,  more  than 
one  anchor  may  apply  to  a  given  Individual.  For  example,  on  a  BARS 
scale  presented  in  Cascio  (1982,  p.  329),  four  of  the  nine  ancnor 
statements  could  conceivably  apply  to  the  same  individual: 


Scale  Value 


Anchors 


"Could  be  expected  to  go  back  on  a  promise 
to  an  individual  who  he  had  told  could 
transfer  back  into  previous  department  if 
she/he  didn't  like  the  new  one.” 


"Could  be  expected  to  remind  sales  personnel 
to  wait  on  customers  instead  of  conversing 
with  each  other." 


"Could  be  expected  never  to  fail  to  conduct 
training  meetings  with  his  people  weekly  at 
a  scheduled  hour  and  to  convey  to  them 
exactly  what  he  expects." 


"Could  be  expected  to  conduct  a  full  day's 
sales  clinic  with  two  new  sales  personnel 
and  thereby  develop  them  into  top  sales 
people  in  the  department.” 


Rating  format  development  variations  with  BARS  can  impact  on  the 
degree  of  psychometric  error  and,  therefore,  users  who  might  be 
tempted  to  vary  the  established  procedures  (perhaps  to  simplify 
them),  should  be  cautioned  not  to  do  so  without  evaluating  the 
impact.  For  example,  Rumsey  and  Mietus  (in  press)  found  that  the 
absence  of  features  such  as  rating  clarification  statements, 
grouping  of  items  by  dimension,  and  overall  evaluation  by  dimension, 
can  significantly  reduce  rating  accuracy. 


CONCLUSIONS 

Nine  prescriptive  themes  for  more  effective  Federal  PA  systems  emerged  from 
a  review  of  the  literature  and  Interviews  with  subject-matter  experts.  These 
themes  are  based  on  opinion  as  well  as  empirical  evidence,  but  they  represent 
the  collective  wisdom  of  those  knowledgeable  in  the  field. 

The  themes  discussed  above  were  presented  one  by  one,  but  in  practice, 
they  require  integration  to  make  the  PA  system  work.  Addressing  any  one,  to 
the  exclusion  of  others,  lessens  the  chances  for  success.  For  example,  the 
recommendation  to  make  the  system  easy  to  use  was  expressed  so  frequently  by 
interviewees  that  it  was  tempting  to  present  it  first  in  the  discussion  of 
themes  Instead  of  last.  Yet,  the  first  two  recommendations  were  to  ensure 
that  the  PA  system  meets  legal  requirements  and  that  it  is  seen  a6  important 
and  worthwhile  by  the  users.  If  the  system  is  easy  to  use  but  not  legal,  it 
is  useless,  and  if  users  see  no  purpose  or  advantage  to  PA,  any  amount  of 
effort  devoted  to  it  will  be  perceived  as  too  much. 

Therefore,  the  first  theme  emphasized  the  primary  reason  for  the  current 
structure  of  PA  systems  In  the  Federal  sector — it  is  required  by  law. 

However,  the  mere  existence  of  a  system  which  meets  legal  specifications  is 
insufficient  to  ensure  usage  beyond  minimal  compliance.  Thus,  the  themes 
which  followed  emphasized  the  steps  that  are  needed  to  make  the  system  work 
as  intended: 

o  As  with  any  new  program  that  requires  the  participation  of  many,  the 
program  must  be  “sold."  With  PA  this  step  is  especially  crucial. 

It  is  highly  likely  that  many  users  at  all  levels  of  the 
organization  already  have  negative  feelings  about  the  process  and 
its  worth. 

o  In  selling  the  program,  It  is  important  to  start  at  the  top  of  the 
organization  because  top  management  plays  a  vital  role  in  ensuring 
the  program's  success.  Top  management  must  both  communicate  its 
commitment  to  developing  and  maintaining  an  effective  PA  system, 
and  obtain  the  commitment  of  the  remainder  of  the  organization. 

Top  management  should  assign  the  responsibility  for  promoting  the 
PA  system  to  a  respected,  high-level  line  manager  who  can  devote 
considerable  time  to  ensuring  that  the  system  be  made  to  work.  The 
emphasis  is  on  line-management  sponsorship  over  personnel  department 
sponsorship  in  order  to  establish  system  credibility  with 
lower-level  managers  and  supervisors. 

o  In  order  for  supervisors  to  be  sold  on  the  PA  process,  they  must 
come  to  view  it  as  an  integral  part  of  their  jobs.  They  must 
understand  that  PA  is  part  of  a  year-round  management  process  which 
includes  Identification  of  goals,  planning  to  accomplish  them, 
coaching,  monitoring  and  documenting  performance,  and  providing 
feedback. 
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o  The  next  step  is  for  top  management  to  identify  specific 

organizational  goals  or  objectives  (to  be  accomplished  in  a  specified 
period  of  time,  such  as  1  to  4  years)  with  measurable  outcomes.  And, 
it  should  emphasize  the  linkage  of  PA  to  the  accomplishment  of  these 
goals.  Supervisors  need  to  communicate  to  their  subordinates  the 
connection  between  the  work  they  are  assigned  to  do  and  the 
accomplishment  of  these  organizational  goals. 

o  Unless  the  PA  system  is  designed  to  take  into  account  idiosyncrasies 
of  the  organization  such  as  size,  dispersion,  and  diversity, 
supervisors  and  managers  will  have  a  hard  time  accepting  that  the 
system  is  supposed  to  help  them  accomplish  organizational  goals. 
Adaptation  of  the  PA  system  to  organizational  needs  is  important. 
Managers  and  supervisors  should  take  an  active  role  in  this  process. 

o  The  PA  system  should  be  designed  so  that  commitment  to  it  will  be 
maintained  beyond  the  initial  introductory  stage.  This  requires 
on-site  managers  who  demonstrate  by  example  their  commitment  to  the 
system,  and  train  other  managers  in  the  skills  required  to  plan, 
coach,  and  provide  feedback  on  performance. 

o  Training  should  be  conducted  for  a  variety  of  purposes  (e.g.,  to  teach 
knowledge  of  the  system,  to  teach  skills  such  as  planning,  coaching, 
and  providing  feedback)  and  for  different  audiences  (e.g., 
supervisors,  subordinates,  military  supervisors  of  civilians).  The 
majority  of  the  training  should  be  conducted  in-house  by  appropriate 
role  models.  Managers  and  supervisors,  as  opposed  to  personnelists , 
should  conduct  most  of  the  training. 

o  Finally,  the  system  should  be  made  as  easy  to  use  as  possible. 
Although  at  first  glance,  the  above  recommendations  may  appear  to 
entail  even  more  effort  than  is  commonly  exerted  to  make  PA  systems 
work,  they  also  may  have  the  effect  of  easing  some  of  the  problems. 

When  supervisors  know  that  the  time  they  spend  on  the  PA  process  is 

valued  by  top  management  as  part  of  the  job,  they  will  be  less 

resentful  of  the  time  it  takes.  If  they  find  the  process  can  be 

useful  in  facilitating  the  accomplishment  of  organizational  goals, 
they  will  be  further  motivated  to  make  it  work.  And,  with 
appropriate  training,  they  will  find  the  system  easier  to  use. 
Further,  such  features  as  a  simplified  means  for  updating 
performance  plans  and  cooperative  development  of  model  performance 
plans  should  ease  the  burden  as  well. 

In  sura,  the  advice  of  Glickman  given  in  1964  still  applies:  If  you  are 
searching  for  a  PA  system  that  is  easily  understood,  does  not  require  much 
time  or  training  to  use,  and  which  the  appraiser  generally  feels  comfortable 
using,  then  "you  have  described  a  relatively  superficial  and  ineffective 
performance  appraisal  6ystem--one  which  is  not  considered  very  important  by 
the  parties  Involved — one  which  could  be  eliminated  without  any  great 
deterioration  in  the  personnel  management  program  of  which  it  was  a  part.  A 
good  performance  appraisal,  one  that  provides  useful  information  to  the 
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employee,  to  his  supervisors,  and  to  management,  takes  considerable  effort  on 
the  part  of  all  of  these  parties  to  develop  and  use  ef f ec t 1 vel y . . .Thu s ,  in  an 
appraisal  system,  as  In  most  human  enterprises,  you  get  what  you  are  willing 
to  pay  for  there  are  no  cheap  but  practical  performance  appraisal  systems" 
(pp.  28-29). 
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appendix  a 

EXAMPLE  OF  AGENCY  STATEMENT  OF  GOALS  AND  SUPPORT  FOR  MBO 


THE  SECRETARY  OF  COMMERCE 

WeihtnQion.  D  C.  ?0?30 

MAR  2  9  1385 


MEMORANDUM  FOR  ALL  EMPLOYEES 

SUBJECT:  Departmental  Goals  for  FY  1985-1988 


For  the  past  year.  Deputy  Secretary  Brown  and  the  Department's  managers  have  been 
assessing  how  well  we  have  done  in  meeting  goals  and  objectives  established  for 
the  first  four  years  of  the  Administration  and  considering  the  need  for  new  and 
revised  goals.  I  am  pleased  to  provide  you  with  the  results  of  that  process:  a 
listing  (attached)  of  new  major  Department  of  Commerce  goals  and  subgoals  for  the 
period  1985-1988. 

These  new  goals  accomplish  several  important  things: 

o  They  restate  Departmental  policies  and  priorities.  Some  of  the  goals  we  set 
four  years  ago  have  been  achieved;  others  have  been  overtaken  by  external 
events.  Now  there  are  new  goals  that  we  can  expect  to  achieve  with  the 
authorities  that  we  have  and  the  resources  we  can  expect  to  get. 

o  They  state  our  aims  more  specifically  than  the  earlier  goals. 

o  They  cut  across  organizational  lines  more  than  before.  This  will  improve 
communication  and  coordination  among  operating  units  with  conroon  interests 
and  responsibilities. 

o  They  cover  more  programs  than  the  earlier  goals.  Four  years  ago  goals  were 
set  only  for  those  areas  where  major  changes  were  envisioned.  Our  managers 
now  feel  that  we  should  have  goals  for  all  of  our  activities. 

I  am  proud  of  the  job  that  our  management  team  has  done  in  setting  these  new 
goals.  They  provide  a  solid  policy  framework  for  planning  and  budgeting  over  the 
next  four  years.  I  am  also  proud  of  the  M30  planning  and  management  system  that 
we  have  implemented  to  manage  our  activities.  The  Commerce  system  is  serving  as  a 
model  for  several  other  agencies  who  are  using  it  to  implement  similar  systems. 

The  system  has  my  full  support  and  the  support  of  my  management  team.  We  are 
counting  on  each  employee  to  continue  supporting  this  system  throughout  the 
planning  period  and  to  continue  their  excellence  in  achieving  the  new  goals  and 
objectives. 


Secretary  of  Commerce  * 


Attachment 
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Increase  America's  Competitiveness  in  the  World  Economy 

•  Increase  U.S.  exports  of  goods  and  services 

•  Expand  foreign  markets  by  removing  barriers  to  travel,  trade  and  investment 

•  Increase  the  number  of  small  and  medium  sized  exporters 

•  Increase  U.S.  share  of  the  international  travel  market 

•  Increase  exports  and  domestic  consumption  of  U.S.  fisheries  products 

•  Increase  the  acceptance  of  international  technical  standards  based  on  U.S.  technology 

•  Strengthen  worldwide  protection  of  intellectual  property 

•  Increase  the  competitiveness  of  U.S.  telecommunications,  computer  and  information 
products  and  services 


Safeguard  Ihe  National  Interest  through  Effective  Administration  of  U.S.  Trade 
Laws 

•  Administer  and  enforce  U.S.  trade  laws  for  reasons  of  national  security,  foreign 
policy,  and  short  supply 

•  Reduce  export  licensing  time  to  an  average  of  30  days  by  1990 

•  Expedite  decontrol  of  non-critical  commodities 

•  Take  prompt,  aggressive  action  against  unfair  trade  practices 


Stimulate  Productivity  and  Economic  Development 

•  Promote  private  sector  capital  formation,  job  creation,  and  new  and  expanding 
industries 

•  Promote  the  development  and  application  of  science  and  technology  in  U.S. 
industries 

•  Reduce  regulatory  and  administrative  barriers  to  innovation  and  growth  of  U.S. 
business  and  industry 

•  Encourage  the  creation  and  expansion  of  minority  businesses 

•  Reduce  patent  pendency  to  18  months  by  1987  and  improve  patent  quality 

•  Promote  the  development  and  growth  of  oceanic  and  atmospheric  industries 

•  Modernize  domestic  statutes  to  reflect  the  changing  nature  of  the  world  economy 


Manage  Effectively  the  Nation's  Oceanic  and  Atmospheric  Resources 

•  Improve  federal  management  of  U.S.  ocean  and  coastal  resources 

•  Provide  timely  environmental  data,  information  and  assessments  to  U.S.  industry 

•  Determine  the  characteristics  and  resources  of  the  U.S.  200  mile  exclusive 
economic  zone  by  1990 

•  Increase  our  understanding  of  oceanic  and  atmospheric  processes  through 
monitoring  and  research 

Improve  Service  Delivery  and  Internal  Management 

•  Provide  accurate  and  timely  information  and  services  to  U.S.  travel  and  trade 
exporters 

•  Provide  more  timely  and  accurate  weather  forecasts  and  warning  services  to  the 
U.S.  public 

•  Restructure  and  modernize  the  national  weather  system  by  1990 

•  Improve  delivery  of  ocean  information  products  and  services 

•  Improve  the  Federal  Government's  use  of  computer  and  telecommunications 
technology 

•  Automate  the  patent  and  trademark  processes  by  1990 

•  Make  Departmental  laboratories  available  for  proprietary  research 

•  Improve  regional  delivery  of  administrative  services 

•  Manage  Departmental  functions  to  reduce  competition  with  the  private  sector 
and  generate  cost  recovery  from  users 
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EXAMPLE  OF  STATEMENT  OF  TOP  MANAGEMENT  SUPPORT  FOR  TRAINING 


DEPARTMENT  OF  THE  ARMY 

HCAOOUAATenS.  U.  S.  AftUY  UATtfliCL  COUUANO 
»OOt  ElJCHMOWCR  AVtNUC.  ALCXAMORIA.  VA  22)U-OOOt 


AMCPE-CE-L  30  January  1985 

SUBJECT:  AMC  Program  on  Improving  Performance  Standard* 


Co  inlander 


1.  On*  of  ay  primary  concern*  in  Che  area  of  personnel  management  i*  the 
quality  of  performance  standard*  in  AMC.  Personnel  Management  surveys  by 
OPM,  BQDA,  AMC  and  AMC  MSC's  continue  to  find  that  performance  standard* 
need  improvement.  Additionally,  inflated  rating  profiles  indicate  to  me 
that  either  performance  standards  are  poor  or  evaluations  are  inaccurate 
or  both. 

2.  I  am  asking  each  AMC  Commander  and  Activity  Chief  Co  take  a  personal 
interest  in  improving  performance  standards  within  his/her  command.  In 
support  of  thi*  effort  HQ  AMC  has  developed  Jointly  with  OPM  a  course  of 
instruction  on  establishing  individual  performance  plana.  We  have  invited 
each  AMC  activity  with  a  CPO  to  name  one  personnelist  and  one  other  person, 
preferably  a  line  manager,  to  receive  the  basic  instruction  and  to  train 
supervisors  upon  return  to  the  installacion/activity .  Five  workshops  are 
scheduled  through  February  1985  to  train  the  90  instructor*. 

3.  The  local  training  is  designed  to  be  given  on  three  consecutive  days. 

We  are  not  requiring  that  every  civilian  and  military  supervisor  take  thi* 
training  regardless  of  his/her  expertise.  However,  we  have  seen  no  ex¬ 
amples  of  standards  to  date  which  would  indicate  that  a  high  level  of 
expertise  exists.  If  it  does  exist,  we  would  recommend  that  those  who 
possess  it  become  a  resource  for  the  local  training  program  at  the  earliest 
point.  We  would  expect  most,  if  not  all.  supervisors  of  civilians  to  attend 
this  training. 

A.  Tf  this  training  is  attended  and  the  principles  covered  are  applied, 

1  ••  cercsin  that  better  performance  standards  will  reeult  and  the  invest¬ 
ment  of  our  supervisors'  time  will  be  well  spent.  Please  give  me  your 
support  in  this  important  effort. 


B  - 1 


